System and Methods to use context graphs for targeting communications

ABSTRACT

This disclosure provides a system and method to use context graphs for targeting communications to a user of an image rendering device. According to one exemplary embodiment, the method provides targeted communications generated as a function of one or more attributes associated with a user requested printed document. The attributes are provided by accessing a context graph including a plurality of links between a plurality of entities.

This application claims the benefit of priority to U.S. ProvisionalApplication No. 61/345,289, filed May 17, 2010, entitled “SYSTEM ANDMETHODS TO USE IN-PARTITE SCALE-FREE GRAPHS FOR INTERPRETING CONTEXTUALINFORMATION AND TARGETING,” by Gnanasambandam et al., U.S. ProvisionalApplication No. 61/345,340, filed May 17, 2010, entitled “OPTIMALAUCTION MECHANISM FOR MULTI-LEVEL DEVICE CLICK-THROUGH (DCT) IN TARGETEDPRINT COMMUNICATION,” by Lee et al., U.S. Provisional Application No.61/345,301, filed May 17, 2010, entitled “SYSTEM AND METHOD TO PRODUCEAND CONTROL SUBSIDIZATION OF TARGETED MATERIALS AT POINT OF SALE,” byGnanasambandam et al., and U.S. Provisional Application No. 61/345,377,filed May 17, 2010, entitled “METHOD FOR IDENTIFICATION, CATEGORIZATIONAND SEARCH OF GRAPHICAL, AUDITORY AND COMPUTATIONAL PATTERN ENSEMBLES,”by Gnanasambandam et al., each of which is hereby incorporated byreference in their entirety.

CROSS REFERENCE TO RELATED PATENTS AND APPLICATIONS

-   U.S. patent application Ser. No. 13/100,636, filed May 4, 2011,    entitled “Method of Providing Targeted Communications to a User of a    Printing System,” by Lee et al.-   U.S. patent application Ser. No. 13/104,136, filed May 10, 2011,    entitled “System and Method to Produce and Control Subsidization of    Targeted Materials at Point of Sale,” by Gnanasambandam et al.-   U.S. patent application Ser. No. ______, filed ______, entitled    “Method for Identification, Categorization and Search of Graphical,    Auditory and Computational Pattern Ensembles,” by Gnanasambandam et    al., each of which is hereby incorporated by reference in their    entirety.

BACKGROUND

This disclosure relates generally to a system and method for generatingtargeted advertisements, and more particularly to a system and methodfor generating targeted advertisements delivered via a printer assemblyincluding at least one printer device. It is currently known foradvertisers to use publicly available information for targetingpotential and publicly available clients. Additionally, advertisers mayuse the publicly available information for advertising merchandise thatis likely to be of interest to a particular person.

INCORPORATION BY REFERENCE

The following patent applications, Patent Application Publications andNon-patent references are incorporated herein by reference in theirentirety:

-   Gnanasambandam et al., U.S. patent application Ser. No. ______,    filed ______, entitled “System and Method to Produce and Control    Subsidization of Targeted Materials at Point of Sale”;-   Gnanasambandam et al., U.S. patent application Ser. No. ______,    filed ______, entitled “Method for Identification, Categorization    and Search of Graphical, Auditory and Computational Pattern    Ensembles”;-   Lee et al., U.S. patent application Ser. No. 13/100,636; filed May    4, 2011, entitled “Method of Providing Targeted Communications to a    User of a Printing System”;-   Gnanasambandam et al., U.S. patent application Ser. No. 12/780,543,    filed May 14, 2010, entitled “System and Method to Prearrange    Hyper-Local Value-Added Marketing Campaigns and Communication Along    Consumer Trajectories”;-   Gnanasambandam, U.S. patent application Ser. No. 12/780,267, filed    May 14, 2010, entitled “In-Situ Mobile Application Suggestions and    Multi-Application Updates Through Context Specific Analytics”;-   Gnanasambandam et al., U.S. Patent Application Publication No. US    2010/0088178 A1, published Apr. 8, 2010, entitled “System and Method    for Generating and Verifying Targeted Advertisements Delivered Via a    Printer Device”;-   Gross, U.S. Patent Application Publication No. US 2010/0005486 A1,    published Jan. 7, 2010, entitled “Apparatus and Method for Embedding    Commercials”;-   Evevsky, U.S. Patent Application Publication No. US 2009/0313060 A1,    published Dec. 17, 2009, entitled “System and Method for    Personalized Printing and Facilitated Delivery of Personalized    Campaign Items”;-   Chow et al., U.S. Patent Application Publication No. US 2009/0157650    A1, published Jun. 18, 2009, entitled “Outbound Content Filtering    Via Automated Inference Detection”;-   Chow et al., U.S. Patent Application Publication No. US 2009/0150365    A1, published Jun. 11, 2009, entitled “Inbound Content Filtering Via    Automated Inference Detection”;-   Gnanasambandam, U.S. patent application Ser. No. 12/761,985, filed    Apr. 16, 2010, entitled “System and Method for Providing Feedback    for Targeted Communications”;-   Liu, U.S. Patent Application Publication No. US 2011/0096354 A1,    published Apr. 28, 2011, entitled “System and Method for Handling    Print Requests from a Mobile Device”;-   Liu et al., U.S. Patent Application Publication No. US 2011/0040823    A1, published Feb. 17, 2011, entitled “System and Method for    Communicating with a Network of Printers Using a Mobile Device”;-   Harrington, U.S. Patent Application Publication No. US 2011/0029952    A1, published Feb. 3, 2011, entitled “Method and System for    Constructing a Document Redundancy Graph”;-   Gnanasambandam et al., U.S. Patent Application Publication No. US    2010/0325422 A1, published Dec. 23, 2010, entitled “System and    Method for Policy-driven File Segmentation and Intercloud File    Storage and Retrieval”;-   Partridge et al., U.S. Patent Application Publication No. US    2010/0309503 A1, published Dec. 9, 2010, entitled “Method and System    for Printing Documents from a Portable Device”;-   Gnanasambandam et al., U.S. Patent Application Publication No, US    2010/0268591 A1, published Oct. 21, 2010, entitled “System and    Method for Selectively Controlling the Use of Functionality in One    or More Multifunction Devices and Subsidizing Their Use Through    Advertisements”;-   Gnanasambandam et al., U.S. Patent Application Publication No. US    2010/0264214 A1, published Oct. 21, 2010, entitled “Method and    System for Providing Contract-free ‘pay-as-you-go’ Options for    Utilization of Multi-function Devices”;-   St. Jacques, Jr. et al., U.S. Patent Application Publication No. US    2010/0149572 A1, published Jun. 17, 2010, entitled “Method and    System for Automatically Providing for Multi-point Document Storing,    Access, and Retrieval”;-   Edelman, B. and M. Schwarz “Optimal Auction Design in a Multi-Unit    Environment: The Case of Sponsored Search Auctions,” Unpublished    manuscript, Harvard Business School 2006;-   Edelman, B., Ostrovsky, M., and Schwarz, M., “Internet Advertising    and the Generalized Second-Price Auction: Selling Billions of    Dollars Worth of Keywords,” American Economic Review, American    Economic Association, Vol. 97(1), pages 242-259, March 2007; Google    AdWords Select, 2002    https://www.google.co/accounts/ServiceLogin?service=adwords&hl;-   Myerson, R. B. “Optimal Auction Design,” Mathematics of Operations    Research, 6(1), pages 58-73, 1981;-   Shazam Entertainment Ltd., 2002,    http://www.shazam.com/music/web/pages/background.html,-   Talluri, K. and van Ryzin, G., “The Theory and Practice of Revenue    Management,” Publisher Springer, 1^(st) Edition, Feb. 23, 2005;-   Ulku, L. “Optimal Combinatorial Mechanism Design,” Unpublished    Manuscript, Rutgers University, 2006;-   Walsh, T., “Search on High Degree Graphs,” In Proceedings of    IJCAI-2001, pages 266-271, 2001.

BRIEF DESCRIPTION

In one embodiment of this disclosure, described is a method of providinguser requested printed material and one or more targeted communicationsto a user of a printing system, the method comprising the printingsystem acquiring material to be printed on a printing device to producethe user requested printed material, the printing device associated withthe printing system; the printing system acquiring the one or moretargeted communications associated with the user requested printedmaterial; and, the printing system printing the user requested printedmaterial utilizing the printing device and providing the acquired one ormore targeted communications to the user, wherein the one or moretargeted communications are generated as a function of one or moreattributes associated with one or more of the user requested printedmaterial the printing device and the user, the one or more attributesprovided by a context graph including a plurality of links between aplurality of entities related to one or more of the user requestedprinted material, the printing device and the user.

In another embodiment, described is a printing system for providing userrequested printed material and one or more targeted communications to auser of the printing system comprising a printing device; and, one ormore servers operating connected to the printing device, wherein one ormore of the printing device and the one or more servers are configuredto execute a method comprising the printing system acquiring material tobe printed on the printing device to produce the user requested printedmaterial; the printing system acquiring the one or more targetedcommunications associated with the user requested printed material; and,the printing device printing the user requested printed material and theprinting device providing the acquired one or more targetedcommunications to the user, wherein of one or more attributes associatedwith one or more of the user requested printed material, the printingdevice and the user, the attributes provided by a context graphincluding a plurality of links between a plurality of entities relatedone or more of the user requested printed material, the printing deviceand the user.

In still another embodiment of this disclosure, described is a method ofproviding a targeted communication to a first image rendering deviceoperatively connected to one or more of a second image rendering deviceand comprising the image rendering device acquiring the targetcommunication from one of the second image rendering device and theserver; and the image rendering device rendering the targetcommunication on the first image rendering device, wherein the targetedcommunication is generated as a function of one or more attributesassociated with one or more of the first image rendering device, and auser associated with the first image rendering device, the one or moreattributes provided by a context graph including a plurality of linksbetween a plurality of entities related to one or more of the firstimage rendering device, the second image rendering device, and the userassociated with the first image rendering device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 and 2 are tripartite context graphs according to an exemplaryembodiment of this disclosure.

FIG. 3 is a 5-partite context graph according to an exemplary embodimentof this disclosure.

FIG. 4 illustrates keyword interconnections associated with k1, k2, k3,k4, k5, k6, k7, k8, k9 and k10 as shown in the context graph of FIG. 3.

FIG. 5 is an exemplary embodiment of a context graph showinginteractions among entities, for example, but not limited to, keywordsand devices.

FIG. 6 shows a power-law in degree distribution of a context graph.

FIG. 7 shows un-weighted context graph A including a scaling exponent of2.58.

FIG. 8 shows a weighted context graph A including a scaling exponent of1.61.

FIG. 9 shows an un-weighted context graph B including a scaling exponent2.6.

FIG. 10 shows a weighted context graph B including a scaling exponent2.29.

FIG. 11 shows a sub-graph including a sequence of edges.

FIG. 12 shows a sub-graph including a loop.

FIG. 13 shows a sub-graph including a single-hop hub where k₃ is theorigin.

FIG. 14 shows a sub-graph including a two hop hub where k₃ is theorigin.

FIG. 15 shows a sub-graph including a 5-partite graph where k₃ is theorigin.

FIG. 16 schematically illustrates a system for context graph basedtargeting using a MFD according to an exemplary embodiment of thisdisclosure.

FIG. 17 shows a targeted communication produced according to anexemplary embodiment of this disclosure, the targeted communicationincluding text ads with graphics from two of the advertisers.

FIG. 18 shows another targeted communication produced according to anexemplary embodiment of this disclosure, the targeted communicationincluding categorized coupons.

FIG. 19 schematically illustrates Transactional Print Workflow accordingto an exemplary embodiment of this disclosure.

DETAILED DESCRIPTION

This disclosure provides systems and methods of using n-partite graphsin determining contextual information that can be used to characterize auser. Such characterizations can be used for personalization andtargeting of information, such as advertisements, to the user. Examplesfor the nodes of the graph include keywords extracted from the titles ofdocuments being printed, the user who prints the document and the deviceon which it is printed. Graph edges are entered connecting the documentkey words to the user and the user to the printing device, but otheredges such as a connection between key words that occur together in atitle can be entered as well. The disclosed systems and methods maintaina large graph that captures all the activities (e.g. document printing),but can extract sub-graphs for a user and uses the information withinthe sub-graph to target advertisements to the user.

Conventional targeting techniques have been largely keyword based orspatio-temporal without specific attention to expressivity that anordering or structure of events can bring to the effectiveness oftargeting. Extracting an expressive sub-graph from a large graph(e.g. >10̂6 nodes and 10̂9 edges) from an n-partite graph of events hasnot been well studied.

Capturing multi-dimensional tuples of information and theinterconnections between events is structurally complex. More so becausea large number of users are causing or contributing to these usersdirectly or through the usage of devices (mobile, print or otherwise).These can be captured coherently by graph theoretical structures.

More information can be gained if the structure of these complexinterconnections (i.e. in the way contextual information gets relatedamongst users, devices, locations, time and other attributes) isanalyzed dynamically. This disclosure also discloses processes toperform that analysis.

In this disclosure, a method is provided wherein graph theoreticstructures in a large graph or sub-graph thereof are utilized to expresscertain correlated user activities and/or event correlations so thatthey can enhance targeted communications to the users. In other words,graphical structures such as sub-graphs—chains of related nodes, loops,paths etc. and statistics on top these structures—degrees, hop-distance,edge weight, edge direction etc. can capture certain inherentconnectivity and association related properties in a more fine grainedfashion. These structures or the properties that govern them can be usedto relate how the users interact with their environments and/or socialgroups. Of particular interest, at least, are the following types ofcontextual information and their n-partite graphical representation whena person interacts with a device: the person, keywords, categories ortopics that relate such an interaction, social group of the person,spatio-temporal aspects of the interaction, usage frequency andcombinations thereof. The disclosed system and method capture and storethese associations in real-time and represents them as n-partite graphs.Subsequently, sub-graphs are extracted dynamically according to a methoddescribed herein. The overall context graph resulting from thecollection and storage of the aforementioned attributes is huge e.g.millions of vertices and edges. Also provided is a method tocharacterize the context graph and methods to use the graph with theaforementioned targeted communication application.

Complex graphs can capture interactions between events and contextualinformation that relate to a large user-population as and when the usersinteract with the environment and the multitude of devices contained inthem, for example, but not limited to, phones, e-readers, printers,kiosks and the like. The scale of the interactions while huge can depictsome underlying structure. As will be described below, provided are someexamples of such interactions exhibiting specific graph theoreticproperties, such as the presence of hubs, scale-free nature, etc. Suchproperties and other structural elements can be utilized for targetingusers with information and marketing material of various kinds.

Examples of Targeting Context Derived from Devices and Processes

Mobile, Phone, E-reader Devices, Printing Devices, InterconnectedPrinting Device (or MFD) Workflows etc.: Meta-data associated with theuse of a device or a multitude of devices can provide targetedcommunications. While some devices have access to global positioningsystem coordinates (latitude, longitude), and cell/base station, othershave sensor readings (accelerometer, magnetometer etc), and context fromdocuments/web applications (or apps). One or more subsets of these canbe utilized for automatically providing meaningful information to theusers of the devices and processes.

Printed documents: When users print to devices (or use other types ofdevices in myriad ways), there is a lot of unused context. In the printworld, this context includes keywords in the document name, location,time, user_id, device name, device capabilities used and such.Furthermore, when a group of people utilize a group of devices, there isyet another social dimension to context—such as the user names of allpeople printing to a given device, document details and keywordscontained in document names, preferred printing times of people andsuch.

Transactional Prints: Credit card statements are one example oftransactional documents. These contain not just spatio-temporalinformation such as when and where the goods and services were purchasedand used, but also the relative importance of such goods to the users.This is valuable information which can be characterized as certain kindsof purchases or user behavior being adjacent to each other in time orspace.

The above applications and combinations thereof are areas where targetedcommunications fit in well. However, these are only provided asexemplary embodiments of the present disclosure and it is to beunderstood that the disclosed system and method is not limited to theseapplications.

Substantially provided herein, at least, is a (1) system and method toextract, rank and unify sub-graphs from large n-partite graphs; (2)system and method to utilize the extracted sub-graph for targeting; (3)controlling the scalability of the system by search terminationstrategies and graph properties; and (4) methods to search and aggregatemultiple types of targeted communication by dynamically relatingmeta-data from context graph with a content repository.

Multi-Partite Context Graph

Multi-partite context graphs can represent a multitude of entities(explicitly or inferentially revealed)—including and not limited toevents, people, devices, locations, date-time, keywords, categories,preferences etc. These connections between the entity types indicateactions performed by one or more users as dictated by a design choice.Furthermore, the weights of connections of entities indicate frequencyof involvement of entities in an action.

For purposes of this disclosure, the meaning of a context graph isprovided herein through an example in the document printing world,although the application of context graphs as provided herein is easilyextended to other domains. In the printing world, context in the form ofkeywords, time-stamps, locations and various subject categories, such asautomobiles, banks, investments etc., are generated collectively by agroup of people by the act of printing, or using other renderingdevices. The extracted keywords may be from job names or from documentcontent. These attributes associated with the printed document can beused to illustrate the concept of a context graph as shown in FIG. 1.

With reference to FIG. 1, a tripartite graph is shown including a set ofkeywords K extracted from attributes associated with the printeddocument, such as the name of the document and other related informationwith or without extracting information from the document, a set ofpeople P, and a set of devices D. Each link in the graph shows one oftwo things—either a person p_(i) used a keyword k_(i) or a person used adevice d_(i). Each unique p_(i), k_(i) or d_(i) appears only once in thegraph. Furthermore, the edge weights are utilized to denote thefrequency of the aforementioned semantics relating to the usage.

Associating Meanings with Sub-Graphs and/or the Whole Graph

With reference to FIG. 2, shown is a graph including additional moresemantics—if a person used two keywords in one usage of the device, suchas one printout, one fax or scan etc., a link is provided between thetwo keywords as shown in the top layer. Note that FIG. 2 is not thetransitive closure of FIG. 1. Notably, such relationships that can bederived with transitive closure are left as a matter of choice andsomething that may be done on the fly for relatively simplerelationships or offline for relatively lengthier chains to which moresemantics may be assigned.

This concept of tripartite context graphs is easily extended ton-partite graphs that can describe more semantics in addition to theabove. For example, FIG. 3 shows the added set of C and N to FIG. 1,where c_(i) may indicate the type of usage such as printing or faxingand n_(i) indicates a feature corresponding to the devices. Graph-basedmodels are chosen for the expressivity for the complex webs ofinformation that can associated with keyword usage and contextgeneration. But as for the term context graphs in the reminder of thisdisclosure, the disclosure does not differentiate between graphs thatare n-partite, their transitive closures or something in between asshown in FIG. 2. FIG. 4 illustrates keyword interconnections associatedwith k1, k2, k3, k4, k5, k6, k7, k8, k9 and k10 as shown in the contextgraph of FIG. 3.

Characterizing Context Graph Structure

The structure and properties of a context graph are fundamental to thedisclosed system and method, initially explained are some findingsrelated to the graphs examined. Context generation has yielded severallarge graphs with heavy keyword interactions that are shown in FIG. 5,according to one example. Entities that are not associated with othersby the users show up as isolated points. NOTE: Each “o” is a discreteentity.

The example of FIG. 5, as well as others not illustrated, from thedevice world of printers show the presence of hubs. These hubs show thatthere are some entities in an action that are heavily connected. Forexample, characterized are two examples. A first example with ˜2600entities (˜3.38 million feasible edges) denoted as graph A, and a secondexample with ˜4200 nodes (˜8.82 million feasible edges) denoted as graphB. The edges further have weights based on the frequency with whichpairs of entities, i.e. keywords, devices or both, were selected incombination. The graph as illustrated in FIG. 6, actually exhibitspower-law characteristics for its degree distribution i.e. if Xrepresents the random variable degree of a given node, P(X≧x) actuallyrepresents a power-law of the form x^(−α). More precisely, the power lawis believed to have an exponential tail in some cases that causes thebehavior to taper off faster at higher degrees.

The power-law coefficients can be separately computed for the weighted(x^(−α1)) and un-weighted (x^(−α2)) graphs. A large difference mayindicate that there is a significant positive impact that is created byfactoring the repetitive social behavior i.e. clustering around commonkeywords, i.e. jargons, in an organization. For example, FIG. 7 showsun-weighted context graph A including a scaling exponent of 2.58; FIG. 8shows a weighted context graph A including a scaling exponent of 1.61;FIG. 9 shows an un-weighted context graph B including a scaling exponent2.6; and FIG. 10 shows a weighted context graph B including a scalingexponent 2.29.

In some cases, such as Graph B, the weighted context graph exhibits anexponential tail as shown in FIG. 10. In other words, P(X≧x) isproportional to x^(−α) e^(−βx) where the exponential term overwhelms thepower law for a large x. During the analysis of contextual informationin an organizational context, it is plausible that the mass usage ofjargons and terms trail off more rapidly beyond a certain level ofpopularity, perhaps due to group size, seasonal effects etc. While ithas been observed that such faster than power-law trail off phenomenondoes occur, the exact causes are not the subject of this disclosure.This disclosure, and the system and method provided herein leverage thedetected properties. One example of the property detected beingpower-law structure and faster than power-law trail off in regions ofthe graph.

Method for Sub-Graph Extraction from Context Graph

In this section, described are various ways sub-graphs can be extractedfrom a context graph to provide targeting. In addition, described arethe generation of the context graph and how it is used in real-time.

Step 1: Graph Maintenance

Initially, a context tuple, e.g. a row of comma-separated context, isreceived by a targeting engine, and each of the entity types in thetuple is recognized by the context graph algorithm associated with thetargeting engine. Either explicitly from data or as the result of aninference, the different entities associated with the context tuples areclassified. The inference can be based on a machine learning algorithm,for example a known supervised classification technique such as thosebased on Support Vector Machines or Singular Value Decomposition. Afterthe context tuple's various entities are recognized, they are assignedto one of the n layers of the n-partite graph and edges and weights areupdated.

Further semantics may be assigned as an extension for each given tuple.The direction of each edge may be altered given the transaction'sdependency relationships or ordering. It is the responsibility of thetuple provider to specify the directionality or ordering of the entitiesat the time the context is provided for producing a targeted message.Each side of edge has a directionality probability which is an aggregateof the set (or any reasonable sub-set) of users assigned to the graph.So an edge directionality probability is denoted e(p_(a), p_(b), p_(ab))where p_(a)+p_(b)+p_(ab)=1, and {a,b} refers to the edge between node aand node b. The graph itself is stored on any scalable platform such asa distributed database. According to one exemplary embodiment, anon-relational distributed database, namely Hbase can be used.

Step 2: Retrieving a Sub-graph and Finding the Dominant Sub-graph

Various graphical structures such as sub-graphs—chains of related nodes,loops, paths, hubs etc. & statistics on top of these structures—degrees,hop-distance, edge weight, edge direction etc. can capture certaininherent connectivity and association related properties in a finegrained fashion. These structures or the properties that govern themrelate to how the users interact with their environments and/or socialgroups, as it is the activities that are utilized to make the graph inthe first place. Of particular interest are at-least the following typesof contextual information and their n-partite graphical representationassociated with a person interacting with a device: the person,keywords, categories or topics that relate such an interaction, socialgroup of the person, spatio-temporal aspects of the interaction, usagefrequency and combinations thereof.

Retrieving a sub-graph relates to extracting a relative smaller graph,i.e. sub-graph, where the keyword from the query is contained in adesignated portion of the sub-graph e.g. center, 2^(nd) layer etc. Sucha keyword is called the origin (w.r.t. to the sub-graph retrieval).According to one exemplary embodiment, the graphical substructures shownin FIGS. 10-15 are of particular interest without any limitations onother substructures that may be added.

Finding a dominant sub-graph relates to ranking the retrieved sub-graphsto find the most suitable sub-graph. Essentially a metric is computedwith the properties of these sub-graphs to determine a quantitativeranking. The metric can be based on degree, edge weight, edge direction,number of nodes in a loop, hop count etc. Other constraints can beassociated with a specific targeting purpose. Usually one of thekeywords, denoted the origin, will be contained in the sub-graphs. Ifmultiple keywords are part of a query, then multiple sub-graphs areretrieved iteratively. The resulting set of dominant sub-graphs are thenmerged based on common nodes. If there is no common node between a pairof origin keywords, the dominant sub-graphs are kept as disjoint piecesat this stage.

Step 3: Utilizing Structure of the Overall Context Graph and Sub-Graphs

For sub-graphs that do not have any overlapping entities, the structureof the overall graph is used. The overall context graph is known toexhibit hubs as previously discussed with regard to the power-lawstructure. Therefore, the strategy is to seek out hubs to connect theseemingly disjoint sub-graphs. The overall context graph is also knownto exhibit other characteristics around the path lengths between nodesNotably, diameter and path lengths can be characterized as otherrelevant metrics.

Initially, the nearest hub associated with the sub-graph is determined.In most cases, this will be just a few hops away because of thescale-free structure. In other cases, the search is terminated owing tointractability. In this case, both sub-graphs are used to identify thenearest hub(s). The hubs may have some latent relationship with thesub-graphs. The revised unified sub-graph is the two sub-graphs joinedby the hub-node. In this way, two seemingly dissimilar entities such asMalaria and CNN for example, can be connected, perhaps because of theairing of a Malaria program on CNN. Such connections become importantwhen two sub-graphs seem very different. Notably, the above strategy isonly done to discover a set of latent connections between sub-graphs.

Often owing to the scale of the context graph stopping criteria forsearch and traversal becomes important. The following are someconditions that may be used: hop-count from origin (most common);reversal in edge direction, hitting a hub, accrued edge weight alongdirection of traversal, number of layers of the graph traversed etc.Notably, sometimes Step 3 may be skipped without any loss of continuityto the method.

Step 4: Using Sub-Graph Outputs for Targeting

Once the sub-graphs are unified or kept disparate, a query isconstructed and sent to an aggregator of information. The query may bebroken into sub-queries to reflect the different ways to offerimportance to the structure of the sub-graph(s) identified. The simplestway is to send the high-degree nodes from the sub-graph along with themost recent input that was received for targeting. Another way is tosend all the nodes in the sub-graph in a sequence and to prioritize theresults that contain the origin entities, e.g. keywords. The sub-graphwill dictate the order of the sub-queries that are iteratively made tothe aggregator.

Step 5: Retrieving Relevant Documents using Sub-graph Output

In the previous step, mentioned was sending a query, possibly composedof sub-queries, to an aggregator of information, e.g. coupons,advertisements, news feeds etc. Checking the relevance of retrieveddocuments is performed in this step. Relevance is performed by countingthe number of entities that exist in the result and matching it with thesub-graph that is delineated in Steps 2-3. For result documents that arein text form the relevance is computed directly by keyword matches. Forimage/audio/video results the relevance is computed on meta-data, tags,transcriptions, translations etc. The filtered and ranked results arereturned to the user in the format of choice. An alternate way is to usethe redundancy observed in retrieved documents for the purpose ofranking. See U.S. patent application Ser. No. 12/533,901, by Harrington,entitled ‘method and system for constructing a document redundancygraph,” filed Jul. 31, 2009.

Implementation Details

In this section, briefly outlined is how to build the context graph andhow to search and maintain it. Explained also is how an incoming job canbe monetized with advertisements and credit is stored for variousverification actions by the user. In the development of this system,utilized was the open source Hadoop/MapReduce/Hbase framework from theApache Foundation to aid in computation and prototyping.

“Printing” is an overloaded term and can be meant to interpret sending adocument or piece of information from one device to another.

Step 1: User prints (or scans faxes) to/from an MFD. Each MFD receivesthe metadata associated with the print (or other use). Each MFD isusually a data node of the Hadoop cluster. Upon receipt of metadata, theMFD calls contextual_ad_fetch( ) which is a remote call to the servercomponent, typically residing with the name-node of the Hadoop filesystem. The MFD that receives the print job/stream caches the jobtemporarily so that the advertisements can be fetched.

Step 2: contextual_ad_fetch( ) first calls ContextGraph_Create_Maintain(). As every user prints (or uses the network of MFDs), the keywords andcontext identifiers (time of day, location, username etc.) areextracted. Now is introduced an edge of vertex in the graph contextgraph G(E,V) where E is the set of edges and V is the set of vertices.Note that weights of the edges have to be recomputed as well.

a. For each context item that is extracted, update the n-partite graph.Generally speaking there may be n types of context. In the prototype,assumed was a tripartite graph with the constituent part being keywords,usernames and devices.

b. If any keyword is not present in the graph, it is added as a newvertex. Likewise for username and device.

c. If the keyword is already present, added is an edge between theusername that printed it and the keyword printed. Also, an edge is addedbetween the username and the device that the job was printed on. Theabove process is repeated for all context items that are extracted. Upto this step, nodes and vertices of the tripartite graphs have beencreated.

d. Next, a link is introduced between each pair of keywords associatedwith the current job. This means that the user associated those keywordstogether and attached some semantics to it in current domain of theusers' operation. According to this example, only the job name was usedfor this step. Other similar metadata from the document may be extractedfrom the document, but only sparingly owing to tractability issues. Thiscreates links in the keyword layer on the fly.

e. Then for each edge that got newly created, the weight is set to one.If there is an edge already between the two vertices under considerationthen the weight is increased by one. The latter condition occurs whensuch a linkage was established by another user or the very same user asthe current user in a prior print job. In this way, the graph iscreated, updated and maintained after every job.

f. From a storage perspective, the graph is stored as co-keywords,co-user-device and co-user-keywords where together with the edgeweights, provided is a social context graph. The graph creation overheadis the same as update overhead as the graph creation is done as severaliterative updates. Update overhead is estimated as follows. Complexityof finding an item in the ordered list is O(log n). Adding a new elementthat is not there is O(1). If the element is already there, computing orupdating the weights requires another O(log n). This process has to bedone for about 5 keywords on average. So the update complexity is10*O(log n) where n is the total number of keywords.

Step 3: contextual_ad_fetch( ) then calls ad_fetch( ) based uponinformation in the graph, i.e. history, and those in the currentcontext, i.e. current job. The aim of this step is to maximize thechance of advertisements and then getting the most relevantadvertisements. The advertisements could be obtained by contacting anadvertisement aggregator. Since it is not a human that is activelyquerying the advertisement aggregator, it is important to send a list ofmost relevant context.

Step 4: Once the ads are obtained from the advertisement aggregator, theavailable real-estate, both electronic and paper, is populated with thebest advertisements. The advertisements that essentially have greaterrelevance, for example, computed relevance based on the occurrence ofqueried keywords in the advertisements, are essentially populated atmore conspicuous areas such as the banner pages and the UI (UserInterface) of the device. Once the ad-filled real-estate is prepared,the rest of document that was cached in step 1 is printed along with theads and/or coupons. According to a prototype, it takes about 2-4 secondsto fetch the targeted ads. An additional couple of seconds may berequired to meld the ads with the customer job in novel ways.

Step 5: Some advertisements may also be sent to the users' desktopsthrough the print-driver so that they can be clicked on or used at alater point. Alternately, some advertisements may be cached at thedevice for use at a subsequent time. If an advertisement is clicked uponat the users' workstations, the advertisement identifier and user arecaptured and transmitted to the device. The device(s) track and storethis information and from time-to-time transmit it to the advertisementaggregator. Printed coupons or other hard copies of advertisements thatthe users can carry with them to a store or vendor include a barcode,micro-text or glyph embedded in the image, so that a vendor can trackthe source of the advertisement. These methods enable the tracking ofprinted or non-printed targeted messages and provide verification to theadvertiser or advertisement aggregator.

A diagrammatic representation of one system according to this disclosureis shown in FIG. 16.

Step 1. According to this embodiment, a user 150 prints a document.Printing a document includes using any device that may have a so called“print button” to invoke the print action. Information includingname/login of person printing, credentials, location (GPS coordinates),print account details, preferences on the printer, preferences for thedocument to be printed including number of pages or page ranges,color/monochrome, document content (words and images with the document)are provided when invoking the print operation. The GUI may contain theprint button and collect these pieces explicitly according to an opt-inmodel with respect to the information. Also the information may bestored in a profile page on a server. Content internal to the documentthat is being submitted for printing may be restricted from use by thecustomer that owns the print job. Printing can also mean printing to adriver such as, but not limited to, an Adobe PDF Printer which willperform the document conversion service by accepting the print job.

Step 2. A service (on the cloud, network or any other known service suchas CUPS—Common Unix Printing Service) takes the printed job and otherinformation from the user that is printing.

Step 3a. Automatic ad fetch and ranking service.

First context is extracted. The service will extract the context (suchas the examples about the user, device, location, time, document above)from the print job. From the context it applies some business rules oralgorithms to extract the query that will be used for fetching adsautomatically. An optimal subset of the context is used for generating aquery. This subset is a function of time, location, user, the device,the service provider (i.e. whether the printing service is in theairport vs. mall vs. office vs. home), past history of printing etc.Relevant context is extracted in the form of a subset of data. Many suchcandidate subset may be unearthed.

Step 3b. This context subset search is facilitated by the context graphthat is stored in a database (the well known hBase in our case), wherehistory of prior context are stored as a context graph explained before.

After the subset is extracted, it is sent to an ad-server that willprovide some ads.

Step 4. Once the ads are received back by the ad-fetch and rankingservice 154, the service performs relevance ranking according tosettings and business rules within the ad fetch and ranking service. Theresults are called relevant advertisements or messages. These are passedback to (152) the print server.

Step 5. (152) This step merges the document to be printed (or convertedto other forms if printing really means document conversion) with adsand messages and then sends them to another application. The otherapplication can be, for example, a phone, an email server, an e-bookreader or in our embodiment a printer or printer GUI.

Step 6. Depending on the device click-through is defined. Click-throughare actions relating to clicks or click-like or view-like activity thatis recorded and transmitted to the advertiser that sent the ads to theads-fetch and ranking service (154). Click through only relates to thoseindividual ads or messages that were acted upon by the user in some way.

List of locations and real-estate (paper and non-paper)

1. Banner page

2. Several Places within Document (margins, sides, interspersing etc.)

3. MFD UI−EIP+Screen

4. Via Print Driver to User Computer

5. Windows Application including browser

6. Mobile Destinations including e-reader, laptops etc.

FIG. 17 shows a targeted communication produced according to anexemplary embodiment of this disclosure, the targeted communicationincluding text ads with graphics from two of the advertisers.

FIG. 18 shows another targeted communication produced according to anexemplary embodiment of this disclosure, the targeted communicationincluding categorized coupons.

FIG. 18 refers to an one embodiment of a personalized ad or couponlisting in a categorized fashion that is again custom generated for theuser or customer. FIG. 18 may be printed or may viewed on a GUI orviewing device, and provides coupons or ads as images relating to 4categories.

(32) and (34)—Business A and Business B coupons 32 and 34 are in theclothes/shoes category.

Business C and Business D coupons 36 and 38 are in a different vacationcategory.

Both the categories chosen and the individual deals/ads chosen arepersonalized to the user view of this listing. Service 154 of FIG. 16generated this listing.

The personalized ad/coupon listing also contains some imagespersonalized to the user. The content within the picture may be enhancedto appeal to the user—such by including text or faces of people known touser or color preferences etc. Also included are tracking elements forpaper or electronic forms such that an authorized device can use thetracking material (such as bar codes or QR codes) and know who utilizedthe personalized ad or coupon.

FIG. 19 shows another use-case in a production printing environmentwhere the aforementioned method can be utilized. Alternately, the pdfscould be just viewed on screen as opposed to printing.

Described is a large workflow with targeted personalization inserted into the workflow stream through the context graph technology.

180: Online or printed statement generation requests are initiated. Thisstarts the process of providing credentials, transaction information(such as those in the statements) provided to the printer/electronicpublisher.

182: The web (182) sends extracted context (context refers to contentlike that explained in FIG. 16 but extracted per user or per transactionfor a user—usually one line in say a credit card statement) and send to(184) a targeting service (equivalent to ad-fetch and ranking service154 in FIG. 16).

184: The process fetches the ads per user per transaction and ranks therelevance and applicability of the ads

186: The process merges ads and per user content into the stream. Streamis said here because in the workflow we have many statementscorresponding to a multitude of users. 186 also applies image colormanipulations or document conversions or device format conversions forrich look and appeal w.r.t device or document format.

188: Useful information, recommendations, coupons, ads or messages thatare generated by 186 are transmitted by web to produce a new stream 190

190: A new print stream is augmented with extra content.

It will be appreciated that variants of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be combined intomany other different systems or applications. Various presentlyunforeseen or unanticipated alternatives, modifications, variations orimprovements therein may be subsequently made by those skilled in theart.

1. A method of providing user requested printed material and one or moretargeted communications to a user of a printing system, the methodcomprising: a) the printing system acquiring material to be printed on aprinting device to produce the user requested printed material, theprinting device associated with the printing system; b) the printingsystem acquiring the one or more targeted communications associated withthe user requested printed material; and, c) the printing systemprinting the user requested printed material utilizing the printingdevice and providing the acquired one or more targeted communications tothe user, wherein the one or more targeted communications are generatedas a function of one or more attributes associated with one or more ofthe user requested printed material the printing device and the user,the one or more attributes provided by a context graph including aplurality of links between a plurality of entities related to one ormore of the user requested printed material, the printing device and theuser.
 2. The method according to claim 1, wherein the context graph is an-partite scale-free graph.
 3. The method according to claim 2, whereinthe context graph includes a historical representation of the pluralityof links between the plurality of entities.
 4. The method according toclaim 3, the plurality of entities associated with one or more ofevents, people, devices, locations, date, time, keywords, categories,preferences.
 5. The method according to claim 3, wherein eh contextgraph includes weights associated with the plurality of links, theweights related to a frequency of links between a pair of entities. 6.The method according to claim 3, wherein the context graph isdynamically updated.
 7. The method according to claim 3, wherein the oneor more attributes are provided by one or more sub-graphs extracted fromthe context graph.
 8. The method according to claim 7, wherein the oneor more sub-graphs extracted from the context graph are associated witha predefined structure.
 9. The method according to claim 7, wherein theone or more attributes are provided by a dominant sub-graph associatedwith the one or more sub-graphs extracted from the context graph. 10.The method according to claim 3, wherein the one or more attributes areprovided by two or more sub-graphs connected by one or more hubsassociated with the context graph.
 11. The method according to claim 1,wherein a verification action is performed by the user, the verificationaction indicating the user has read at least one target communication.12. A printing system for providing user requested printed material andone or more targeted communications to a user of the printing systemcomprising: a printing device; and, one or more servers operatingconnected to the printing device, wherein one or more of the printingdevice and the one or more servers are configured to execute a methodcomprising: the printing system acquiring material to be printed on theprinting device to produce the user requested printed material; a) theprinting system acquiring the one or more targeted communicationsassociated with the user requested printed material; and, b) theprinting device printing the user requested printed material and theprinting device providing the acquired one or more targetedcommunications to the user, wherein of one or more attributes associatedwith one or more of the user requested printed material, the printingdevice and the user, the attributes provided by a context graphincluding a plurality of links between a plurality of entities relatedone or more of the user requested printed material, the printing deviceand the user.
 13. The printing system according to claim 12, wherein thecontext graph is a n-Partite scale-free graph.
 14. The printing systemaccording to claim 13, wherein the context graph includes a historicalrepresentation of the plurality of link between the plurality ofentities.
 15. The printing system according to claim 14, the pluralityof entities associated with one or more of events, people, devices,locations, date, time, keywords, categories, preferences.
 16. Theprinting system according to claim 14, wherein the context graphincludes weights associated with the plurality of links, the weightsrelated to a frequency of links between a pair of entities.
 17. Theprinting system according to claim 14, where the context graph isdynamically updated.
 18. The printing system according to claim 14,wherein the one or more attributes are provided by one or moresub-graphs extracted from the context graph.
 19. The printing systemaccording to claim 18, wherein the one or more sub-graphs extracted fromthe context graph are associated with a predefined structure.
 20. Theprinting system according to claim 18, wherein the one or moreattributes are provided by a dominant sub-graph associated with the oneor more sub-graphs extracted from the context graph.
 21. The printingsystem according to claim 14, wherein the one or more attributes areprovided by two or more sub-graphs connected by one or more hubsassociated with the context graph.
 22. The printing system according toclaim 12, wherein a verification action is performed by the user, theverification action indicating the user has read at least one targetedcommunication.
 23. A method of providing a targeted communication to afirst image rendering device operatively connected to one or more of asecond image rendering device and comprising: a) the image renderingdevice acquiring the target communication from one of the second imagerendering device and the server; and b) the image rendering devicerendering the target communication on the first image rendering device,wherein the targeted communication is generated as a function of one ormore attributes associated with one or more of the first image renderingdevice, and a user associated with the first image rendering device, theone or more attributes provided by a context graph including a pluralityof links between a plurality of entities related to one or more of thefirst image rendering device, the second image rendering device, and theuser associated with the first image rendering device.
 24. The methodaccording to claim 23, wherein the context graph is a n-Partitescale-free graph.
 25. The method according to claim 23, wherein thecontext graph includes weights associated with the plurality of links,the weights related to a frequency of links between a pair of entities.